A well-known encoding system for compression recording of moving images is H.264/MPEG-4 AVC (hereinafter referred to as H.264). In H.264, to improve the encoding efficiency, possible systems are intra-frame prediction encoding (hereinafter referred to as “intra prediction”) and inter-frame prediction encoding (hereinafter referred to as “inter prediction”). The intra prediction is prediction encoding with using the correlation between pixels in a picture. The inter prediction is prediction encoding with using the correlation between pixels between pictures with motion vectors. In the inter prediction, the motion vectors are searched. The motion vectors represent a magnitude of motion from a reference image in the unit blocks to be encoded to a target image to be encoded. The searched motion vectors are encoded similarly to residual prediction for the target image, and multiplexed as a part of an encoded stream output from an image encoding apparatus. The above contents are disclosed in Japanese Unexamined Patent Application Publication No. 2015-2371.